Convolutional Neural Networks


In this notebook, we train a CNN to classify images from the CIFAR-10 database.

The images in this database are small color images that fall into one of ten classes; some example images are pictured below.

Test for CUDA

Since these are larger (32x32x3) images, it may prove useful to speed up your training time by using a GPU. CUDA is a parallel computing platform and CUDA Tensors are the same as typical Tensors, only they utilize GPU's for computation.


In [1]:
import torch
import numpy as np

# Check if CUDA is available
train_on_gpu = torch.cuda.is_available()

if not train_on_gpu:
    print('CUDA is not available.  Training on CPU ...')
else:
    print('CUDA is available!  Training on GPU ...')


CUDA is available!  Training on GPU ...

In [2]:
# Forcing to train on the CPU
train_on_gpu = False

Load the Data

Downloading may take a minute. We load in the training and test data, split the training data into a training and validation set, then create DataLoaders for each of these sets of data.


In [3]:
from torchvision import datasets
import torchvision.transforms as transforms
from torch.utils.data.sampler import SubsetRandomSampler

# Number of subprocesses to use for data loading
num_workers = 0
# How many samples per batch to load
batch_size = 20
# Percentage of training set to use as validation
valid_size = 0.2

# Convert data to a normalized torch.FloatTensor
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(mean=(0.5, 0.5, 0.5),
                         std=(0.5, 0.5, 0.5))
    ])

# Choose the training and test datasets
train_data = datasets.CIFAR10(root='data',
                              train=True,
                              download=True,
                              transform=transform)
test_data = datasets.CIFAR10(root='data',
                             train=False,
                             download=True,
                             transform=transform)

# Obtain training indices that will be used for validation
num_train = len(train_data)
indices = list(range(num_train))
np.random.shuffle(indices)
split = int(np.floor(valid_size * num_train))
train_idx, valid_idx = indices[split:], indices[:split]

# Define samplers for obtaining training and validation batches
train_sampler = SubsetRandomSampler(indices=train_idx)
valid_sampler = SubsetRandomSampler(indices=valid_idx)

# Prepare data loaders (combine dataset and sampler)
train_loader = torch.utils.data.DataLoader(dataset=train_data,
                                           batch_size=batch_size,
                                           sampler=train_sampler,
                                           num_workers=num_workers)
valid_loader = torch.utils.data.DataLoader(dataset=train_data,
                                           batch_size=batch_size,
                                           sampler=valid_sampler,
                                           num_workers=num_workers)
test_loader = torch.utils.data.DataLoader(dataset=test_data,
                                          batch_size=batch_size, 
                                          num_workers=num_workers)

# Specify the image classes
classes = ['airplane', 'automobile', 'bird', 'cat', 'deer',
           'dog', 'frog', 'horse', 'ship', 'truck']


Files already downloaded and verified
Files already downloaded and verified

Visualize a Batch of Training Data


In [4]:
import matplotlib.pyplot as plt
%matplotlib inline

# Helper function to un-normalize and display an image
def imshow(img):
    img = img / 2 + 0.5  # Unnormalize
    plt.imshow(np.transpose(img, (1, 2, 0)))  # Convert from Tensor image

In [5]:
# Obtain one batch of training images
dataiter = iter(train_loader)
images, labels = dataiter.next()
images = images.numpy() # convert images to numpy for display

# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(25, 4))
# Display 20 images
for idx in np.arange(20):
    ax = fig.add_subplot(2, 20/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])
    ax.set_title(classes[labels[idx]])



In [6]:
images[0].shape


Out[6]:
(3, 32, 32)

View an Image in More Detail

Here, we look at the normalized red, green, and blue (RGB) color channels as three separate, grayscale intensity images.


In [7]:
rgb_img = np.squeeze(images[3])
channels = ['red channel', 'green channel', 'blue channel']

fig = plt.figure(figsize = (36, 36)) 
for idx in np.arange(rgb_img.shape[0]):
    ax = fig.add_subplot(1, 3, idx + 1)
    img = rgb_img[idx]
    ax.imshow(img, cmap='gray')
    ax.set_title(channels[idx])
    width, height = img.shape
    thresh = img.max()/2.5
    for x in range(width):
        for y in range(height):
            val = round(img[x][y],2) if img[x][y] !=0 else 0
            ax.annotate(str(val), xy=(y,x),
                    horizontalalignment='center',
                    verticalalignment='center', size=8,
                    color='white' if img[x][y]<thresh else 'black')



Define the Network Architecture

This time, you'll define a CNN architecture. Instead of an MLP, which used linear, fully-connected layers, you'll use the following:

  • Convolutional layers, which can be thought of as stack of filtered images.
  • Maxpooling layers, which reduce the x-y size of an input, keeping only the most active pixels from the previous layer.
  • The usual Linear + Dropout layers to avoid overfitting and produce a 10-dim output.

A network with 2 convolutional layers is shown in the image below and in the code, and you've been given starter code with one convolutional and one maxpooling layer.

TODO: Define a model with multiple convolutional layers, and define the feedforward network behavior.

The more convolutional layers you include, the more complex patterns in color and shape a model can detect. It's suggested that your final model include 2 or 3 convolutional layers as well as linear layers + dropout in between to avoid overfitting.

It's good practice to look at existing research and implementations of related models as a starting point for defining your own models. You may find it useful to look at this PyTorch classification example or this, more complex Keras example to help decide on a final structure.

Output volume for a convolutional layer

To compute the output size of a given convolutional layer we can perform the following calculation (taken from Stanford's cs231n course):

We can compute the spatial size of the output volume as a function of the input volume size (W), the kernel/filter size (F), the stride with which they are applied (S), and the amount of zero padding used (P) on the border. The correct formula for calculating how many neurons define the output_W is given by (W−F+2P)/S+1.

For example for a 7x7 input and a 3x3 filter with stride 1 and pad 0 we would get a 5x5 output. With stride 2 we would get a 3x3 output.


In [8]:
import torch.nn as nn
import torch.nn.functional as F

# Define the CNN architecture
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        # Convolutional layers
        # Input 32x32x3
        self.conv1 = nn.Conv2d(in_channels=3,
                               out_channels=16,
                               kernel_size=(3, 3),
                               stride=(1, 1),
                               padding=1)
        # Input 16x16x16
        self.conv2 = nn.Conv2d(in_channels=16,
                               out_channels=32,
                               kernel_size=(3, 3),
                               stride=(1, 1),
                               padding=1)
        # Input 8x8x32
        self.conv3 = nn.Conv2d(in_channels=32,
                               out_channels=64,
                               kernel_size=(3, 3),
                               stride=(1, 1),
                               padding=1)
        # Max pooling layer
        self.pool = nn.MaxPool2d(kernel_size=(2, 2),
                                 stride=(2, 2))
        # Dropout
        self.dropout = nn.Dropout(p=0.25)
        # Linear Layers
        # Input flatten 4x4x64 -> 500
        self.fc1 = nn.Linear(in_features=4 * 4 * 64,
                             out_features=128)
        # Input 500, Output 10
        self.fc2 = nn.Linear(in_features=128,
                             out_features=10)
        

    def forward(self, x):
        # Add sequence of convolutional and max pooling layers
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = self.pool(F.relu(self.conv3(x)))
        # Flatten the image
        x = x.view(-1, 64 * 4 * 4)
        # First fully-connected layer
        x = F.relu(self.fc1(x))
        # Adding dropout
        x = self.dropout(x)
        # Adding second layer
        x = self.fc2(x)
        return x

# Create a complete CNN
model = Net()
print(model)

# Move tensors to GPU if CUDA is available
if train_on_gpu:
    model.cuda()


Net(
  (conv1): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (conv2): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (conv3): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (pool): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
  (dropout): Dropout(p=0.25)
  (fc1): Linear(in_features=1024, out_features=128, bias=True)
  (fc2): Linear(in_features=128, out_features=10, bias=True)
)

Specify Loss Function and Optimizer

Decide on a loss and optimization function that is best suited for this classification task. The linked code examples from above, may be a good starting point; this PyTorch classification example or this, more complex Keras example. Pay close attention to the value for learning rate as this value determines how your model converges to a small error.

TODO: Define the loss and optimizer and see how these choices change the loss over time.


In [9]:
import torch.optim as optim

# Specify loss function
criterion = nn.CrossEntropyLoss()

# Specify optimizer
optimizer = optim.Adam(params=model.parameters(),
                       lr=0.01)

Train the Network

Remember to look at how the training and validation loss decreases over time; if the validation loss ever increases it indicates possible overfitting.


In [10]:
# Number of epochs to train the model
n_epochs = 50 

# Initialize validation loss
valid_loss_min = np.Inf 

for epoch in range(1, n_epochs+1):

    # Keep track of training and validation loss
    train_loss = 0.0
    valid_loss = 0.0
    
    model.train()
    for data, target in train_loader:
        # Move tensors to GPU if CUDA is available
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        # Clear the gradients of all optimized variables
        optimizer.zero_grad()
        # Forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        # Calculate the batch loss
        loss = criterion(output, target)
        # Backward pass: compute gradient of the loss with respect to model parameters
        loss.backward()
        # Perform a single optimization step (parameter update)
        optimizer.step()
        # Update training loss
        train_loss += loss.item() * data.size(0)
    
    # Validate the model 
    model.eval()
    for data, target in valid_loader:
        # Move tensors to GPU if CUDA is available
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        # Forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        # Calculate the batch loss
        loss = criterion(output, target)
        # Update average validation loss 
        valid_loss += loss.item()*data.size(0)
    
    # Calculate average losses
    train_loss = train_loss/len(train_loader.dataset)
    valid_loss = valid_loss/len(valid_loader.dataset)
        
    # Print training/validation statistics 
    print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
        epoch, train_loss, valid_loss))
    
    # Save model if validation loss has decreased
    if valid_loss <= valid_loss_min:
        print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(
        valid_loss_min,
        valid_loss))
        torch.save(model.state_dict(), './models/model_cifar.pth')
        valid_loss_min = valid_loss


Epoch: 1 	Training Loss: 1.518332 	Validation Loss: 0.437108
Validation loss decreased (inf --> 0.437108).  Saving model ...
Epoch: 2 	Training Loss: 1.451812 	Validation Loss: 0.352971
Validation loss decreased (0.437108 --> 0.352971).  Saving model ...
Epoch: 3 	Training Loss: 1.428917 	Validation Loss: 0.347359
Validation loss decreased (0.352971 --> 0.347359).  Saving model ...
Epoch: 4 	Training Loss: 1.420911 	Validation Loss: 0.339554
Validation loss decreased (0.347359 --> 0.339554).  Saving model ...
Epoch: 5 	Training Loss: 1.400894 	Validation Loss: 0.339719
Epoch: 6 	Training Loss: 1.385790 	Validation Loss: 0.338357
Validation loss decreased (0.339554 --> 0.338357).  Saving model ...
Epoch: 7 	Training Loss: 1.385927 	Validation Loss: 0.350423
Epoch: 8 	Training Loss: 1.376446 	Validation Loss: 0.343328
Epoch: 9 	Training Loss: 1.372045 	Validation Loss: 0.331796
Validation loss decreased (0.338357 --> 0.331796).  Saving model ...
Epoch: 10 	Training Loss: 1.365836 	Validation Loss: 0.342317
Epoch: 11 	Training Loss: 1.366717 	Validation Loss: 0.327667
Validation loss decreased (0.331796 --> 0.327667).  Saving model ...
Epoch: 12 	Training Loss: 1.358536 	Validation Loss: 0.343967
Epoch: 13 	Training Loss: 1.353880 	Validation Loss: 0.327165
Validation loss decreased (0.327667 --> 0.327165).  Saving model ...
Epoch: 14 	Training Loss: 1.354801 	Validation Loss: 0.326904
Validation loss decreased (0.327165 --> 0.326904).  Saving model ...
Epoch: 15 	Training Loss: 1.360357 	Validation Loss: 0.329795
Epoch: 16 	Training Loss: 1.354234 	Validation Loss: 0.327541
Epoch: 17 	Training Loss: 1.351358 	Validation Loss: 0.324181
Validation loss decreased (0.326904 --> 0.324181).  Saving model ...
Epoch: 18 	Training Loss: 1.349943 	Validation Loss: 0.327167
Epoch: 19 	Training Loss: 1.347604 	Validation Loss: 0.333549
Epoch: 20 	Training Loss: 1.346961 	Validation Loss: 0.323800
Validation loss decreased (0.324181 --> 0.323800).  Saving model ...
Epoch: 21 	Training Loss: 1.340147 	Validation Loss: 0.329953
Epoch: 22 	Training Loss: 1.349587 	Validation Loss: 0.330764
Epoch: 23 	Training Loss: 1.344023 	Validation Loss: 0.341974
Epoch: 24 	Training Loss: 1.338906 	Validation Loss: 0.328170
Epoch: 25 	Training Loss: 1.350144 	Validation Loss: 0.333617
Epoch: 26 	Training Loss: 1.347025 	Validation Loss: 0.332653
Epoch: 27 	Training Loss: 1.346253 	Validation Loss: 0.323946
Epoch: 28 	Training Loss: 1.349576 	Validation Loss: 0.332283
Epoch: 29 	Training Loss: 1.355076 	Validation Loss: 0.320347
Validation loss decreased (0.323800 --> 0.320347).  Saving model ...
Epoch: 30 	Training Loss: 1.343924 	Validation Loss: 0.331845
Epoch: 31 	Training Loss: 1.356193 	Validation Loss: 0.331090
Epoch: 32 	Training Loss: 1.342123 	Validation Loss: 0.339415
Epoch: 33 	Training Loss: 1.366292 	Validation Loss: 0.341200
Epoch: 34 	Training Loss: 1.355870 	Validation Loss: 0.323190
Epoch: 35 	Training Loss: 1.360569 	Validation Loss: 0.356290
Epoch: 36 	Training Loss: 1.354508 	Validation Loss: 0.337742
Epoch: 37 	Training Loss: 1.365936 	Validation Loss: 0.329611
Epoch: 38 	Training Loss: 1.350055 	Validation Loss: 0.337826
Epoch: 39 	Training Loss: 1.348939 	Validation Loss: 0.331585
Epoch: 40 	Training Loss: 1.352709 	Validation Loss: 0.327099
Epoch: 41 	Training Loss: 1.360529 	Validation Loss: 0.334959
Epoch: 42 	Training Loss: 1.352216 	Validation Loss: 0.334667
Epoch: 43 	Training Loss: 1.353500 	Validation Loss: 0.326905
Epoch: 44 	Training Loss: 1.353691 	Validation Loss: 0.336539
Epoch: 45 	Training Loss: 1.346189 	Validation Loss: 0.322341
Epoch: 46 	Training Loss: 1.363567 	Validation Loss: 0.331788
Epoch: 47 	Training Loss: 1.362247 	Validation Loss: 0.328470
Epoch: 48 	Training Loss: 1.361150 	Validation Loss: 0.329351
Epoch: 49 	Training Loss: 1.360710 	Validation Loss: 0.330917
Epoch: 50 	Training Loss: 1.351894 	Validation Loss: 0.332326

Load the Model with the Lowest Validation Loss


In [11]:
model.load_state_dict(torch.load('./models/model_cifar.pth'))

Test the Trained Network

Test your trained model on previously unseen data! A "good" result will be a CNN that gets around 70% (or more, try your best!) accuracy on these test images.


In [12]:
# track test loss
test_loss = 0.0
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))

model.eval()
# iterate over test data
for data, target in test_loader:
    # move tensors to GPU if CUDA is available
    if train_on_gpu:
        data, target = data.cuda(), target.cuda()
    # forward pass: compute predicted outputs by passing inputs to the model
    output = model(data)
    # calculate the batch loss
    loss = criterion(output, target)
    # update test loss 
    test_loss += loss.item() * data.size(0)
    # convert output probabilities to predicted class
    _, pred = torch.max(output, 1)    
    # compare predictions to true label
    correct_tensor = pred.eq(target.data.view_as(pred))
    correct = np.squeeze(correct_tensor.numpy()) if not train_on_gpu else np.squeeze(correct_tensor.cpu().numpy())
    # calculate test accuracy for each object class
    for i in range(batch_size):
        label = target.data[i]
        class_correct[label] += correct[i].item()
        class_total[label] += 1

# average test loss
test_loss = test_loss/len(test_loader.dataset)
print('Test Loss: {:.6f}\n'.format(test_loss))

for i in range(10):
    if class_total[i] > 0:
        print('Test Accuracy of %5s: %2d%% (%2d/%2d)' % (classes[i], 100 * class_correct[i] / class_total[i],
                                                         np.sum(class_correct[i]), np.sum(class_total[i])))
    else:
        print('Test Accuracy of %5s: N/A (no training examples)' % (classes[i]))

print('\nTest Accuracy (Overall): %2d%% (%2d/%2d)' % (100. * np.sum(class_correct) / np.sum(class_total),
                                                      np.sum(class_correct), np.sum(class_total)))


Test Loss: 1.601007

Test Accuracy of airplane: 47% (471/1000)
Test Accuracy of automobile: 59% (597/1000)
Test Accuracy of  bird: 38% (388/1000)
Test Accuracy of   cat: 21% (211/1000)
Test Accuracy of  deer: 20% (207/1000)
Test Accuracy of   dog: 28% (280/1000)
Test Accuracy of  frog: 53% (534/1000)
Test Accuracy of horse: 51% (511/1000)
Test Accuracy of  ship: 43% (436/1000)
Test Accuracy of truck: 54% (546/1000)

Test Accuracy (Overall): 41% (4181/10000)

Question: What are your model's weaknesses and how might they be improved?

Answer:

1) Collect more data / Augment the data / Balance the data

2) Test different hyperparamaters / optimizers

3) Try different architecture

Visualize Sample Test Results


In [13]:
# obtain one batch of test images
dataiter = iter(test_loader)
images, labels = dataiter.next()
images.numpy()

# move model inputs to cuda, if GPU available
if train_on_gpu:
    images = images.cuda()

# get sample outputs
output = model(images)
# convert output probabilities to predicted class
_, preds_tensor = torch.max(output, 1)
preds = np.squeeze(preds_tensor.numpy()) if not train_on_gpu else np.squeeze(preds_tensor.cpu().numpy())

# plot the images in the batch, along with predicted and true labels
fig = plt.figure(figsize=(25, 4))
for idx in np.arange(20):
    ax = fig.add_subplot(2, 20/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])
    ax.set_title("{} ({})".format(classes[preds[idx]], classes[labels[idx]]),
                 color=("green" if preds[idx]==labels[idx].item() else "red"))